Thanks for this contribution, I believe it's very valuable to expand the libraries functionality!
One suggestion for an improvement of the mask delineation: Replace the proposed matching of rotated ROI aligned feature maps with hbox mask targets (using rbox2hox in mask_target.py -> mask_target_single & rotated_fcn_mask_head.py -> _do_paste_mask()) with an rotated ROI alignment for the mask targets. Thereby the feature maps as well as the mask targets are actually spatially matching, which allows a better mask prediction. To illustrate the improvement, you can find a comparison of both approaches for the use case of agricultural parcel delineation below.

@zytx121

When I use my own training set, why the box is shifted, but the segm is correct? @zytx121 @fkroeber

@fkroeber Thank you for sharing, the results are much better than now! May I ask if you are willing to provide us with a PR?

Hello,
I am new to MMRotate. I've found this library while I was searching for an instance segmentation model with oriented bounding boxes. Thank you @zytx121, I think this project will solve my problem.
I have seen that DOTA dataset type is privileged in MMRotate. But I guess that masks have to be provided. Therefore, I assume that DOTA dataset is not supported by your project. This is why I ask you what type of dataset have to be used in this project ?